Review for the article "Knowledge Exchange as a Motivational Problem – Results of an Empirical Research Program" by Cress, U. & Kimmerle, J.
In this article the authors study the social dilemma of costs and benefits in knowledge-exchange situations (I will shorten knowledge-exchange as KE in the following). They approach the KE situations in experimental studies where they use a cost-benefit model of material costs and benefits. The authors aim to achieve results that would have relevance in computer-supported information exchange in several types of group situations.
The authors say in the title of this paper that they will present " Results of an Empirical Research Program" which leads me to believe that this is more a review article than a research article, even though they do present some original results here also. This brings me to the main problem of this article. I really had difficulties in figuring out which results were original for this paper and which have been published earlier by these authors as part of this research program.
The other problem I have with this paper is it's structure. If this is meant to be a review article then the structure is ok, except that the review nature of this paper should be made more clear, and the authors could have tried to make a deeper synthesis of their results from the whole research program.
However, I suggest to the authors that they would consider presenting this as a research article, with a formal structure of Introduction, Methods, Results and Discussion, and leave the review / discussion of their previous results to the Discussion part. As such I find this article difficult to read, as the experimental procedures, results from here and earlier and discussion are all mixed together.
For example, at the end of page 2 (by the way, it would have made this review immensely easier if the authors had provided the reader with page and line numbers :) ) there is a description of "The assignment" (Methods) then under same major heading of "Experimental examination..." is a title "The perceived payoff structure" where some results are also presented. Then in the results, as said before, it is not always clear, which results are original for this paper, and which are from earlier studies.
I do believe that the results of this whole research program are interesting and worth publishing, but this paper could be made much better with a clearer focus.
Detailed comments:
page 2, "The assignment": The experimental procedure is not very informatively written. I'd like to know how many groups of 6 people there were in this study and, also, if the participants were the same people in all different kinds of separate experiments. Some information of the background of participants and how they were allocated to groups would be of benefit to the reader, too.
page 3 gives the description of Stages 1 and 2. What was the time between these two stages?
page 3, "The perceived...": As I said earlier, this chapter is a mixture of methods and results. Here I'm also confused weather the results given here are original for this paper or totally from Cress, Kimmerle & Hesse (2006).
page 4, and elsewhere in Results: I'm not sure what is meant with the word 'significant'. Do the authors mean statistically significant, or just some otherwise perceived difference? If the first, as must be the case in Figure 4 (main effects and interactions), then I find it most unscientific that no information about statistical analysis (or other analysis methods) is given. If the latter, Istrongly suggest that some other word that 'significantly' is used.
page 4, "Group size" This experiment comes totally out of blue. Just results are given and nearly nothing of the experimental setting.
page 5, second paragraph, sentence "This assumption was confirmed in various experimental studies' is vague. The references should be given here, and not in the end of this para.
page 7, first line. This is one of the places were I (once again) wondered about original results vs. earlier results from this program. 'a further experiment included a feedback...' Is this 'further experiment' a new one conducted for this article, or from Cress and Kimmerle (2007)? If from C & K, then it is not 'a further' but 'a previous' experiment...
page 7, first paragraph: I did not understand from the description given, whether the participants were shown both the number of their own entries plus groups average, and then they continued with the experiment, or they were given group averages before the experiment.
page 7, third para: As no reference is given I'm thinking this is an original result for this paper, but is it actually from Cress and Kimmerle (2007)... Here some statistical analysis must have been done, but no information is given.
page 7, last whole para. Is this paragraph something that the authors are pondering or is it something someone else has written / studied. If the latter, then some references are needed.
page 8, figure 4. Is this from Cress (2005)? If so, the reference must be given in figure legend, too.
page 8, second last paragraph. This is the only place were the authors consider the possibility that some people pass on of knowledge 'just for the sake of it'. This could be mentioned briefly in introduction too.
Figures:
In figures the authors use the scale 0-1, and in text they use percentages. It is better to use only one or the other in both figures and in text.
The figure legends could be made more informative, as such they do not stand on their own. For example, in Fig. 1. 'Contribution rate' of what? In Fig. 2. it is not clear, just by looking at the figure, that in category 'different cost' higher cost is connected with higher importance. And, finally, in Fig. 4. the words 'Significant main... effect' do not form a sentence.
What if the bars in Fig. 3 were grouped as three groups (of different bonus types) of two bars (of relevance)?, Just a suggestion...
Confidential comments for the editor only.
As I wrote in my comments to authors, I'm no able to see clearly what results are new for this article and what have been published earlier. However, if the authors can rewrite the paper to make this matter clear, either in writing this as a research article on more clearly a review article, then the results do warrant publication. Still, I'd like to see a more synergistic approach when the previous and new results are discussed.
The methods should be explained in more detail. I also have some concerns about the statistical analysis, meaning that it is not obvious if such analysis has been conducted or not. As I'm not very deeply in the field, I'm not familiar enough with the relevant litterature to comment on the references used.
So, I my suggestion is that this paper need such major revision that it should be revised and resubmitted. If this is the editors decision, I'm ready to review the revised paper.
Yours, ...
REVIEW - COMMENTS FROM REFEREES
Review for the article "Knowledge Exchange as a Motivational Problem – Results of an Empirical Research Program" by Cress, U. & Kimmerle, J.
In this article the authors study the social dilemma of costs and benefits in knowledge-exchange situations (I will shorten knowledge-exchange as KE in the following). They approach the KE situations in experimental studies where they use a cost-benefit model of material costs and benefits. The authors aim to achieve results that would have relevance in computer-supported information exchange in several types of group situations.
The authors say in the title of this paper that they will present " Results of an Empirical Research Program" which leads me to believe that this is more a review article than a research article, even though they do present some original results here also. This brings me to the main problem of this article. I really had difficulties in figuring out which results were original for this paper and which have been published earlier by these authors as part of this research program.
The other problem I have with this paper is it's structure. If this is meant to be a review article then the structure is ok, except that the review nature of this paper should be made more clear, and the authors could have tried to make a deeper synthesis of their results from the whole research program.
However, I suggest to the authors that they would consider presenting this as a research article, with a formal structure of Introduction, Methods, Results and Discussion, and leave the review / discussion of their previous results to the Discussion part. As such I find this article difficult to read, as the experimental procedures, results from here and earlier and discussion are all mixed together.
For example, at the end of page 2 (by the way, it would have made this review immensely easier if the authors had provided the reader with page and line numbers :) ) there is a description of "The assignment" (Methods) then under same major heading of "Experimental examination..." is a title "The perceived payoff structure" where some results are also presented. Then in the results, as said before, it is not always clear, which results are original for this paper, and which are from earlier studies.
I do believe that the results of this whole research program are interesting and worth publishing, but this paper could be made much better with a clearer focus.
Detailed comments:
page 2, "The assignment": The experimental procedure is not very informatively written. I'd like to know how many groups of 6 people there were in this study and, also, if the participants were the same people in all different kinds of separate experiments. Some information of the background of participants and how they were allocated to groups would be of benefit to the reader, too.
page 3 gives the description of Stages 1 and 2. What was the time between these two stages?
page 3, "The perceived...": As I said earlier, this chapter is a mixture of methods and results. Here I'm also confused weather the results given here are original for this paper or totally from Cress, Kimmerle & Hesse (2006).
page 4, and elsewhere in Results: I'm not sure what is meant with the word 'significant'. Do the authors mean statistically significant, or just some otherwise perceived difference? If the first, as must be the case in Figure 4 (main effects and interactions), then I find it most unscientific that no information about statistical analysis (or other analysis methods) is given. If the latter, I strongly suggest that some other word that 'significantly' is used.
page 4, "Group size" This experiment comes totally out of blue. Just results are given and nearly nothing of the experimental setting.
page 5, second paragraph, sentence "This assumption was confirmed in various experimental studies' is vague. The references should be given here, and not in the end of this para.
page 7, first line. This is one of the places were I (once again) wondered about original results vs. earlier results from this program. 'a further experiment included a feedback...' Is this 'further experiment' a new one conducted for this article, or from Cress and Kimmerle (2007)? If from C & K, then it is not 'a further' but 'a previous' experiment...
page 7, first paragraph: I did not understand from the description given, whether the participants were shown both the number of their own entries plus groups average, and then they continued with the experiment, or they were given group averages before the experiment.
page 7, third para: As no reference is given I'm thinking this is an original result for this paper, but is it actually from Cress and Kimmerle (2007)... Here some statistical analysis must have been done, but no information is given.
page 7, last whole para. Is this paragraph something that the authors are pondering or is it something someone else has written / studied. If the latter, then some references are needed.
page 8, figure 4. Is this from Cress (2005)? If so, the reference must be given in figure legend, too.
page 8, second last paragraph. This is the only place were the authors consider the possibility that some people pass on of knowledge 'just for the sake of it'. This could be mentioned briefly in introduction too.
Figures:
In figures the authors use the scale 0-1, and in text they use percentages. It is better to use only one or the other in both figures and in text.
The figure legends could be made more informative, as such they do not stand on their own. For example, in Fig. 1. 'Contribution rate' of what? In Fig. 2. it is not clear, just by looking at the figure, that in category 'different cost' higher cost is connected with higher importance. And, finally, in Fig. 4. the words 'Significant main... effect' do not form a sentence.
What if the bars in Fig. 3 were grouped as three groups (of different bonus types) of two bars (of relevance)?, Just a suggestion...
Confidential comments for the editor only.
As I wrote in my comments to authors, I'm no able to see clearly what results are new for this article and what have been published earlier. However, if the authors can rewrite the paper to make this matter clear, either in writing this as a research article on more clearly a review article, then the results do warrant publication. Still, I'd like to see a more synergistic approach when the previous and new results are discussed.
The methods should be explained in more detail. I also have some concerns about the statistical analysis, meaning that it is not obvious if such analysis has been conducted or not. As I'm not very deeply in the field, I'm not familiar enough with the relevant litterature to comment on the references used.
So, I my suggestion is that this paper need such major revision that it should be revised and resubmitted. If this is the editors decision, I'm ready to review the revised paper.
Yours, ...